fix: preserve prepared binary params across retries#25872
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
gouhongshen
left a comment
There was a problem hiding this comment.
Codex automated review
No blocking or non-blocking defects found. Binary metadata is preserved through execution, plan copying, remote serialization, and retry ownership; final cleanup releases owned parameters. Diff check passed and protobuf tests passed; CGo-transitive focused tests were unavailable locally because cgo/libmo.dylib is absent.
aunjgr
left a comment
There was a problem hiding this comment.
P1: Negotiate binary-parameter metadata before remote execution.
The new PrepareParamInfo.is_bin field is additive, so an older CN silently ignores it. That older worker then evaluates ParamExpression without setting Vector.IsBin and treats a binary prepared parameter as text. A new coordinator can still schedule such a query to an old CN during a rolling upgrade, producing incorrect indexed or scanned comparison results with no error.
Please gate remote placement on worker protocol capability whenever any prepared parameter has a binary flag, or preserve the semantics through a representation understood by old workers. Add a new-coordinator to old-worker compatibility regression; decoding an old payload on a new worker covers only the opposite direction.
gouhongshen
left a comment
There was a problem hiding this comment.
Codex automated review
A blocking resource leak exists in the new mixed-version compatibility rejection. Earlier mixed-version correctness concerns are addressed by the current routing guard. Focused tests could not compile because required CGo headers are unavailable.
P1 - Compatibility rejection leaks already-sent RPC futures (pkg/shardservice/service_read.go:144)
Read creates futures and defers only futureSlice.close(), which merely clears pooled references; it does not call Future.Close(). The normal success path closes each future after Get(). If an earlier compatible shard has already been sent and a later shard fails validateRemoteReadCompatibility here, Read returns before draining or closing the earlier future. That future remains registered in morpc's backend future map, and its response/resources can remain retained. Repeated multi-shard binary reads with mixed target versions can therefore leak active futures and response buffers. Validate all remote targets before sending, or explicitly close/drain every already-created future on this error path.
# Conflicts: # pkg/pb/pipeline/pipeline.pb.go # pkg/sql/compile/compile_test.go
gouhongshen
left a comment
There was a problem hiding this comment.
Codex automated review
Reviewed git diff origin/main...HEAD and all existing PR reviews/comments. Prior compatibility, decoded-process ownership, and rejected-RPC cleanup concerns are addressed on the current head; no findings. Focused local tests are blocked by absent CGo third-party headers; current PR checks show no failures.
aunjgr
left a comment
There was a problem hiding this comment.
Reviewed independently with an ownership and retry-generation audit. Prepared vectors now have an explicit owner, are detached only while a retry Compile sharing the Process is released, restored by defer, copied into the receiver mpool before RPC buffers can be reused, and freed by Process.Free. Binary provenance is carried through expression evaluation and guarded at the shard-read version boundary. The stream-reader changes also close both the reader and decoded-process lease exactly once across build failure, explicit close, and expiry, outside the global map lock. Rejected method RPC contexts are canceled on both write success and failure. No actionable issues found.
Merge Queue Status
This pull request spent 1 hour 9 minutes 17 seconds in the queue, including 1 hour 8 minutes 57 seconds running CI. Required conditions to merge
|
What type of PR is this?
Which issue(s) this PR fixes:
issue #25421
What this PR does / why we need it:
Preserves per-parameter binary metadata for user variables passed through SQL
EXECUTE ... USING, including local expression evaluation, plan substitution,deep copies, and remote CN process serialization.
The previous implementation released owned prepared parameters from
Process.Free(). Compile retries reuse the sameProcess, while releasing anintermediate retry
Compilealso callsProcess.Free(). That cleared theparameters before the next retry and could cause a nil pointer panic under
TPCC workloads.
This change detaches the complete prepared-parameter state before releasing an
intermediate retry compile and restores it afterward. The transferred state
includes the parameter vector, per-parameter binary flags, and ownership.
Normal final cleanup still releases owned vectors exactly once.
Validation:
pkg/container/vector,pkg/vm/process,pkg/sql/colexec,pkg/sql/plan,pkg/sql/compile, andpkg/frontendmake buildmake static-check(0 issues)